Laser speckle force feedback estimation

ABSTRACT

Provided herein are systems, methods, and media capable of determining estimated force applied on a target tissue region to enable tactile feedback during interaction with said target tissue region.

CROSS-REFERENCE

This application is a continuation of International Patent ApplicationNo. PCT/US21/12524, filed on Jan. 7, 2021, which claims priority to U.S.Provisional Patent Application No. 62/958,501 filed on Jan. 8, 2020,each of which is incorporated herein by reference in its entirety.

BACKGROUND

While the increased prevalence of machine operated and telemedicalsurgery robots has enabled significant treatment advances through theirhigh precision and low requisite incision areas, many such systems areunable to provide caregivers with tactile feedback. Such tactilefeedback is often useful to determine critical structures duringsurgery.

SUMMARY

Provided herein is a computer-implemented method for determining anestimated force applied on a target tissue region, the methodcomprising: obtaining a set of images of the target tissue region;determining a perfusion property, a set of spatial measurement, or bothof the target tissue region based at least on the set of images;determining a deformation of the target tissue region based at least onthe set of spatial measurements; determining a viscoelastic property ofthe target tissue region based at least on the deformation of the targettissue region, the perfusion property of the target tissue region, orboth; and determining the estimated force applied on the target tissueregion based at least on the viscoelastic property of the target tissueregion.

In some embodiments, the set of images comprises a laser speckle image,an RGB image, an RGB-Depth image, or any combination thereof. In someembodiments, the laser speckle image is a subjective laser speckleimage, an objective laser speckle image, a near-field laser speckleimage, or any combination thereof. In some embodiments, the set ofimages is obtained while emitting two or more different wavelengths oflight at the target tissue region.

In some embodiments, the set of images is obtained while emitting lightat the target tissue region having a number of different wavelengths ofabout 10 to about 1,000. In some embodiments, the set of images isobtained while emitting light at the target tissue region having anumber of different wavelengths of about 10 to about 50, about 10 toabout 100, about 10 to about 200, about 10 to about 300, about 10 toabout 400, about 10 to about 500, about 10 to about 600, about 10 toabout 700, about 10 to about 800, about 10 to about 900, about 10 toabout 1,000, about 50 to about 100, about 50 to about 200, about 50 toabout 300, about 50 to about 400, about 50 to about 500, about 50 toabout 600, about 50 to about 700, about 50 to about 800, about 50 toabout 900, about 50 to about 1,000, about 100 to about 200, about 100 toabout 300, about 100 to about 400, about 100 to about 500, about 100 toabout 600, about 100 to about 700, about 100 to about 800, about 100 toabout 900, about 100 to about 1,000, about 200 to about 300, about 200to about 400, about 200 to about 500, about 200 to about 600, about 200to about 700, about 200 to about 800, about 200 to about 900, about 200to about 1,000, about 300 to about 400, about 300 to about 500, about300 to about 600, about 300 to about 700, about 300 to about 800, about300 to about 900, about 300 to about 1,000, about 400 to about 500,about 400 to about 600, about 400 to about 700, about 400 to about 800,about 400 to about 900, about 400 to about 1,000, about 500 to about600, about 500 to about 700, about 500 to about 800, about 500 to about900, about 500 to about 1,000, about 600 to about 700, about 600 toabout 800, about 600 to about 900, about 600 to about 1,000, about 700to about 800, about 700 to about 900, about 700 to about 1,000, about800 to about 900, about 800 to about 1,000, or about 900 to about 1,000.In some embodiments, the set of images is obtained while emitting lightat the target tissue region having a number of different wavelengths ofabout 10, about 50, about 100, about 200, about 300, about 400, about500, about 600, about 700, about 800, about 900, or about 1,000. In someembodiments, the set of images is obtained while emitting light at thetarget tissue region having a number of different wavelengths of atleast about 10, about 50, about 100, about 200, about 300, about 400,about 500, about 600, about 700, about 800, or about 900. In someembodiments, the set of images is obtained while emitting light at thetarget tissue region having a number of different wavelengths of at mostabout 50, about 100, about 200, about 300, about 400, about 500, about600, about 700, about 800, about 900, or about 1,000.

In some embodiments, the set of images of the target issue region andthe set spatial measurements of the target tissue region are obtainedsimultaneously in real time as the target issue region undergoes thedeformation. In some embodiments, the set of images of the target issueregion is obtained in-vitro. In some embodiments, the set of images ofthe target issue region is obtained in-vivo. In some embodiments, atleast one of the set of images of the target issue region is obtainedwhile the target tissue region undergoes a known deformation by apre-determined force. In some embodiments, the target tissue region is asoft tissue region. In some embodiments, determining the mechanicalproperty, the viscoelastic property, or both of the target tissue regionis performed by a machine learning algorithm. In some embodiments, theviscoelastic property comprises a viscous property, an elastic property,a fluid mechanics property, or any combination thereof. In someembodiments, the method further comprises obtaining depth measurementsfrom a depth sensor, and wherein the deformation of the target tissueregion is further based on the depth measurements. In some embodiments,the spatial measurements are one-dimensional, two-dimensional, orthree-dimensional. In some embodiments, the depth sensor comprises astereo camera, a video camera, a time of flight sensor, or anycombination thereof. In some embodiments, the deformation of the targettissue region comprises a one-dimensional deformation, a two-dimensionaldeformation, a three-dimensional deformation, or any combinationthereof. In some embodiments, determining the estimated force applied tothe target tissue region is performed by a machine learning algorithm.In some embodiments, the force is applied by a human operator, andwherein the method further comprises providing a feedback to theoperator based on the determined estimated force applied on the targettissue region. In some embodiments, the feedback comprises a visualfeedback, an auditory feedback, a haptic feedback, or any combinationthereof. In some embodiments, the visual feedback comprises a colorcoded visual feedback, a displayed value, a map, or any combinationthereof corresponding to the estimated force. In some embodiments, arelationship between the estimated force and the feedback is linear,non-linear, or exponential. In some embodiments, the force is applied byan autonomous or semi-autonomous device, and wherein the method furthercomprises providing a control feedback to the autonomous orsemi-autonomous device based on the force applied by the deformedtissue. In some embodiments, the autonomous or semi-autonomous devicealters its treatment based on the control feedback. In some embodiments,the method further comprises determining a fluid flow rate within thetarget tissue based at least on (i) the set of images, (ii) the spatialmeasurements, (iii) the viscoelastic property of the target tissueregion, (iv) the deformation of the target tissue region, or anycombination thereof. In some embodiments, the fluid is blood, sweat,semen, saliva, pus, urine, air, mucus, milk, bile, a hormone, or anycombination thereof. In some embodiments, the fluid flow rate within thetarget tissue is determined by a machine learning algorithm. In someembodiments, the fluid flow rate is determined by a machine learningalgorithm. In some embodiments, the method further comprises determiningan identification of the target tissue based at least on (i) the set ofimages, (ii) the spatial measurements, (iii) the viscoelastic propertyof the target tissue region, (iv) the deformation of the target tissueregion, or any combination thereof. In some embodiments, theidentification of the target tissue is determined by a machine learningalgorithm. In some embodiments, the identification of the target tissueis an identification that the target tissue is cancerous, benign,malignant, or healthy.

Another aspect provided herein is a computer-implemented systemcomprising: a digital processing device comprising: at least oneprocessor, an operating system configured to perform executableinstructions, a memory, and a computer program including instructionsexecutable by the digital processing device to create an application fordetermining an estimated force applied on a target tissue region, theapplication comprising: a module obtaining a set of images of the targettissue region; a module determining a perfusion property, a set ofspatial measurement, or both of the target tissue region based at leaston the set of images; a module determining a deformation of the targettissue region based at least on the set of spatial measurements; amodule determining a viscoelastic property of the target tissue regionbased at least on the deformation of the target tissue region, theperfusion property of the target tissue region, or both; and a moduledetermining the estimated force applied on the target tissue regionbased at least on the viscoelastic property of the target tissue region.

In some embodiments, the set of images comprises a laser speckle image,an RGB image, an RGB-Depth image, or any combination thereof. In someembodiments, the laser speckle image is a subjective laser speckleimage, an objective laser speckle image, a near-field laser speckleimage, or any combination thereof. In some embodiments, the set ofimages is obtained while emitting two or more different wavelengths oflight at the target tissue region. In some embodiments, the set ofimages is obtained while emitting about 10 to about 1,000 differentwavelengths of light at the target tissue region. In some embodiments,the set of images of the target issue region and the set spatialmeasurements of the target tissue region are obtained simultaneously inreal time as the target issue region undergoes the deformation. In someembodiments, the set of images of the target issue region is obtainedin-vitro. In some embodiments, the set of images of the target issueregion is obtained in-vivo. In some embodiments, at least one of the setof images of the target issue region is obtained while the target tissueregion undergoes a known deformation by a pre-determined force. In someembodiments, the target tissue region is a soft tissue region. In someembodiments, determining the mechanical property, the viscoelasticproperty, or both of the target tissue region is performed by a machinelearning algorithm. In some embodiments, the viscoelastic propertycomprises a viscous property, an elastic property, a fluid mechanicsproperty, or any combination thereof. In some embodiments, theapplication further comprises a module obtaining depth measurements froma depth sensor, and wherein the deformation of the target tissue regionis further based on the depth measurements. In some embodiments, thespatial measurements are one-dimensional, two-dimensional, orthree-dimensional. In some embodiments, the depth sensor comprises astereo camera, a video camera, a time of flight sensor, or anycombination thereof. In some embodiments, the deformation of the targettissue region comprises a one-dimensional deformation, a two-dimensionaldeformation, a three-dimensional deformation, or any combinationthereof. In some embodiments, determining the estimated force applied tothe target tissue region is performed by a machine learning algorithm.In some embodiments, the force is applied by a human operator, andwherein the application further comprises a module providing a feedbackto the operator based on the determined estimated force applied on thetarget tissue region. In some embodiments, the feedback comprises avisual feedback, an auditory feedback, a haptic feedback, or anycombination thereof. In some embodiments, the visual feedback comprisesa color coded visual feedback, a displayed value, a map, or anycombination thereof corresponding to the estimated force. In someembodiments, a relationship between the estimated force and the feedbackis linear, non-linear, or exponential. In some embodiments, the force isapplied by an autonomous or semi-autonomous device, and wherein theapplication further comprises a module providing a control feedback tothe autonomous or semi-autonomous device based on the force applied bythe deformed tissue. In some embodiments, the autonomous orsemi-autonomous device alters its treatment based on the controlfeedback. In some embodiments, the application further comprising amodule determining a fluid flow rate within the target tissue based atleast on (i) the set of images, (ii) the spatial measurements, (iii) theviscoelastic property of the target tissue region, (iv) the deformationof the target tissue region, or any combination thereof. In someembodiments, the fluid is blood, sweat, semen, saliva, pus, urine, air,mucus, milk, bile, a hormone, or any combination thereof. In someembodiments, the fluid flow rate within the target tissue is determinedby a machine learning algorithm. In some embodiments, the fluid flowrate is determined by a machine learning algorithm. In some embodiments,the application further comprising a module determining anidentification of the target tissue based at least on (i) the set ofimages, (ii) the spatial measurements, (iii) the viscoelastic propertyof the target tissue region, (iv) the deformation of the target tissueregion, or any combination thereof. In some embodiments, theidentification of the target tissue is determined by a machine learningalgorithm. In some embodiments, the identification of the target tissueis an identification that the target tissue is cancerous, benign,malignant, or healthy.

Another aspect provided herein is a non-transitory computer-readablestorage media encoded with a computer program including instructionsexecutable by a processor to create an application for determining anestimated force applied on a target tissue region, the applicationcomprising: a module obtaining a set of images of the target tissueregion; a module determining a perfusion property, a set of spatialmeasurement, or both of the target tissue region based at least on theset of images; a module determining a deformation of the target tissueregion based at least on the set of spatial measurements; a moduledetermining a viscoelastic property of the target tissue region based atleast on the deformation of the target tissue region, the perfusionproperty of the target tissue region, or both; and a module determiningthe estimated force applied on the target tissue region based at leaston the viscoelastic property of the target tissue region.

In some embodiments, the set of images comprises a laser speckle image,an RGB image, an RGB-Depth image, or any combination thereof. In someembodiments, the laser speckle image is a subjective laser speckleimage, an objective laser speckle image, a near-field laser speckleimage, or any combination thereof. In some embodiments, the set ofimages is obtained while emitting two or more different wavelengths oflight at the target tissue region. In some embodiments, the set ofimages is obtained while emitting about 10 to about 1,000 differentwavelengths of light at the target tissue region. In some embodiments,the set of images of the target issue region and the set spatialmeasurements of the target tissue region are obtained simultaneously inreal time as the target issue region undergoes the deformation. In someembodiments, the set of images of the target issue region is obtainedin-vitro. In some embodiments, the set of images of the target issueregion is obtained in-vivo. In some embodiments, at least one of the setof images of the target issue region is obtained while the target tissueregion undergoes a known deformation by a pre-determined force. In someembodiments, the target tissue region is a soft tissue region. In someembodiments, determining the mechanical property, the viscoelasticproperty, or both of the target tissue region is performed by a machinelearning algorithm. In some embodiments, the viscoelastic propertycomprises a viscous property, an elastic property, a fluid mechanicsproperty, or any combination thereof. In some embodiments, theapplication further comprises a module obtaining depth measurements froma depth sensor, and wherein the deformation of the target tissue regionis further based on the depth measurements. In some embodiments, thespatial measurements are one-dimensional, two-dimensional, orthree-dimensional. In some embodiments, the depth sensor comprises astereo camera, a video camera, a time of flight sensor, or anycombination thereof. In some embodiments, the deformation of the targettissue region comprises a one-dimensional deformation, a two-dimensionaldeformation, a three-dimensional deformation, or any combinationthereof. In some embodiments, determining the estimated force applied tothe target tissue region is performed by a machine learning algorithm.In some embodiments, the force is applied by a human operator, andwherein the application further comprises a module providing a feedbackto the operator based on the determined estimated force applied on thetarget tissue region. In some embodiments, the feedback comprises avisual feedback, an auditory feedback, a haptic feedback, or anycombination thereof. In some embodiments, the visual feedback comprisesa color coded visual feedback, a displayed value, a map, or anycombination thereof corresponding to the estimated force. In someembodiments, a relationship between the estimated force and the feedbackis linear, non-linear, or exponential. In some embodiments, the force isapplied by an autonomous or semi-autonomous device, and wherein theapplication further comprises a module providing a control feedback tothe autonomous or semi-autonomous device based on the force applied bythe deformed tissue. In some embodiments, the autonomous orsemi-autonomous device alters its treatment based on the controlfeedback. In some embodiments, the application further comprises amodule determining a fluid flow rate within the target tissue based atleast on (i) the set of images, (ii) the spatial measurements, (iii) theviscoelastic property of the target tissue region, (iv) the deformationof the target tissue region, or any combination thereof. In someembodiments, the fluid is blood, sweat, semen, saliva, pus, urine, air,mucus, milk, bile, a hormone, or any combination thereof. In someembodiments, the fluid flow rate within the target tissue is determinedby a machine learning algorithm. In some embodiments, the fluid flowrate is determined by a machine learning algorithm. In some embodiments,the application further comprising a module determining anidentification of the target tissue based at least on (i) the set ofimages, (ii) the spatial measurements, (iii) the viscoelastic propertyof the target tissue region, (iv) the deformation of the target tissueregion, or any combination thereof. In some embodiments, theidentification of the target tissue is determined by a machine learningalgorithm. In some embodiments, the identification of the target tissueis an identification that the target tissue is cancerous, benign,malignant, or healthy.

Another aspect provided herein is a computer-implemented method fortraining a neural network to determine an elastic property of a targetissue region, the method comprising: generating a first training setcomprising a plurality of sets of set of images, wherein each set ofimages comprises a first speckle image of the target issue region atrest and a second speckle image of the target issue region beingdeformed by a known force; training the neural network in a first stageusing the first training set; generating a second training setcomprising the first training set and the sets of set of images whoseelastic property value was incorrectly determined after the first stageof training; and training the neural network in a second stage using thesecond training set. In some embodiments, the set of images comprises asubjective set of images, an objective set of images, a near-field setof images, or any combination thereof. In some embodiments, the set ofimages is obtained while emitting at least 10 different wavelengths oflight at the target tissue region. In some embodiments, the set ofimages is obtained while emitting about 10 to about 1,000 differentwavelengths of light at the target tissue region. In some embodiments,the viscoelastic property comprises a viscous property, an elasticproperty, a fluid mechanics property, or any combination thereof. Insome embodiments, the spatial measurements are one-dimensional,two-dimensional, or three-dimensional.

Another aspect of the present disclosure provides a non-transitorycomputer readable medium comprising machine executable code that, uponexecution by one or more computer processors, implements any of themethods above or elsewhere herein.

Another aspect of the present disclosure provides a system comprisingone or more computer processors and computer memory coupled thereto. Thecomputer memory comprises machine executable code that, upon executionby the one or more computer processors, implements any of the methodsabove or elsewhere herein.

Additional aspects and advantages of the present disclosure will becomereadily apparent to those skilled in this art from the followingdetailed description, wherein only illustrative embodiments of thepresent disclosure are shown and described. As will be realized, thepresent disclosure is capable of other and different embodiments, andits several details are capable of modifications in various obviousrespects, all without departing from the disclosure. Accordingly, thedrawings and description are to be regarded as illustrative in nature,and not as restrictive.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.To the extent publications and patents or patent applicationsincorporated by reference contradict the disclosure contained in thespecification, the specification is intended to supersede and/or takeprecedence over any such contradictory material.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the disclosure are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present disclosure will be obtained by reference tothe following detailed description that sets forth illustrativeembodiments, in which the principles of the disclosure are utilized, andthe accompanying drawings of which:

FIG. 1 shows a schematic diagram of a method for determining anestimated force, per an embodiment herein;

FIG. 2 shows a schematic diagram of a method for training a neuralnetwork to determine a viscoelastic property of a target issue region,per an embodiment herein;

FIG. 3 shows a schematic diagram of various light frequencies, per anembodiment herein;

FIG. 4 shows a schematic diagram of a machine learning algorithm todetermine a viscoelastic property of a target issue region, per anembodiment herein;

FIG. 5A shows an image of a device for obtaining a set of images of thetarget tissue region, per an embodiment herein;

FIG. 5B shows an image of a device with a laparoscope for obtaining aset of images of the target tissue region, per an embodiment herein;

FIG. 6 shows an image of a connectivity device for transferring the setof images of the target tissue region, per an embodiment herein;

FIG. 7 shows an image of a system for collecting and transferring theset of images of the target tissue region, per an embodiment herein;

FIG. 8A shows an image of a sample tissue region;

FIG. 8B shows an image of a sample tissue region injected with an;

FIG. 9A shows another image of a target tissue region, per an embodimentherein;

FIG. 9B shows an image of the perfusion within the target tissue region,per an embodiment herein;

FIG. 9C shows an image of the target tissue region overlaid with theimage of the perfusion within the target tissue region, per anembodiment herein;

FIG. 10A shows an image of an unablated target tissue region injectedwith the ICG dye;

FIG. 10B shows an image of an unablated target tissue region injectedoverlaid with the determined perfusion property, per an embodimentherein;

FIG. 10C shows an image of an ablated target tissue region injected withthe ICG dye;

FIG. 10D shows an image of an ablated target tissue region injectedoverlaid with the determined perfusion property, per an embodimentherein;

FIG. 11 shows an exemplary setup to capture a speckle image of a targettissue region undergoing a known deformation by a pre-determined force,per an embodiment herein; and

FIG. 12 shows a non-limiting example of a computing device; in thiscase, a device with one or more processors, memory, storage, and anetwork interface, per an embodiment herein.

DETAILED DESCRIPTION

As machine operated and telemedical surgery robots and mechanisms areunable to provide caregivers with tactile feedback, there is an unmetneed for systems, methods, and media capable of determining mechanicalproperties of target tissues to enable such feedback. The presentdisclosure addresses at least the above need.

Method, Systems and Media for Determining an Estimated Force

Provided herein is a computer-implemented method, systems and media fordetermining an estimated force applied on a target tissue region. Insome embodiments, per FIG. 1, the method comprises: obtaining a set ofimages of the target tissue region 101; determining a perfusionproperty, a set of spatial measurement, or both of the target tissueregion 102; determining a deformation of the target tissue region 103;determining a viscoelastic property of the target tissue region 104; anddetermining the estimated force applied on the target tissue region 105.In some embodiments, the estimated force applied on the target tissueregion is determined based at least on the viscoelastic property of thetarget tissue region.

In some embodiments, the target tissue is a soft tissue. In someembodiments, the target tissue is an epithelial tissue, connectivetissue, muscular tissue, nervous tissue, or any combination thereof. Insome embodiments, the target tissue region is a treatment regionreceiving treatment by a caregiver. In some embodiments, the targettissue region has an area of about 2 mm², 5 mm², 10 mm², 20 mm², 50 mm²,100 mm², 200 mm², 500 mm², 1,000 mm², 10,000 mm², 100,000 mm², 1,000,000mm², or more including increments therein. In some embodiments, thetarget tissue is in-vitro. In some embodiments, the target tissue isin-vivo.

Perfusion Property

Current methods of determining perfusion in a target tissue, per FIGS.8A and 8B, typically require the infusion of a fluorescent dye (e.g. anindocyanine green (ICG) dye) into a patient. While key perfusionstructures are visible in FIG. 8B, such infusions have severalshortcomings. First as the dye requires about 5 minutes to about 24hours to reach the target tissue, such a procedure must be plannedbefore a surgery of the target tissue, and/or delay the visualizationeffects. Any additional planning and treatment steps that could go awryshould be avoided to ensure a successful surgery. Such a large dyevisualization variation among patients further encumbers its use.Further, as clinicians are charged per dosage of the dye, mistimed oruntimely injections are costly. Second, the visualization capabilitiesof the dye dissipate as it flows through the bloodstream, leaving a verynarrow opportunity of use. Finally, such dyes are not indicated for allpatients based on their biologic interactions.

By contrast, in some embodiments, the methods, systems, and media hereindo not require the use of a dye or other injected visualization medium.Further the methods, systems, and media herein require little to noplanning for use, can be used instantly without any waiting periods, andcan be used continually throughout a surgery without inducing extracosts or procedures.

Further, per FIGS. 10A-D the systems, methods, and media herein are morecapable at determining areas of perfusion property than the currentlyavailable ICG dies. Although visualizations of an unablated targettissue with the ICG dye, per FIG. 10A, and via the instant methods,systems, and media, per FIG. 10B, show the same areas of reducedperfusion 100A and 100B, reduced perfusion area 100C of tissuevisualized with the ICG dye, per FIG. 10C, is incapable of detectingareas of reduced perfusion induced by ablation. By contrast, per FIG.10D, the methods, systems, and media herein are capable of detectingareas of reduced perfusion induced by ablation 110 in addition to theremaining areas of reduced perfusion 110D.

In some embodiments, the perfusion property of the target tissue regionis determined based at least on the set of images. In some embodiments,the perfusion measures a rate at which a fluid is delivered to tissue,or volume of the fluid per unit time per unit tissue mass in m³/(s kg)or ml/min/g. In some embodiments, the fluid is blood, sweat, semen,saliva, pus, urine, air, mucus, milk, bile, a hormone, or anycombination thereof. In some embodiments, the perfusion property isfurther determined by measurements collected by an oximeter, a pulserate monitor, or any combination thereof. In some embodiments, theperfusion property is further determined based on predeterminedperfusion properties of an organ or tissue. FIG. 9A shows an exemplaryimage of a target tissue region. FIG. 9B shows an exemplary image of theperfusion of the target tissue region. FIG. 9C shows an exemplary imageof the target tissue region overlaid with the image of the perfusion ofthe target tissue region. As seen, the ability to see the perfusion ofthe target tissue in addition to its image enables a surgical operatorto determine areas with higher and lower perfusion to treat and/or avoidthose portions the target tissue accordingly. Areas with higherperfusion normally indicate critical structures, which, if damagedduring surgery, can be harmful or fatal to the patient. It is estimatedthat about 2% of hysterectomies result in complications due to suchdamage of critical structure, whereas such complications cost about 1billion dollars to treat.

Spatial Measurements

In some embodiments, the set of spatial measurements of the targettissue region is determined based at least on the set of images. In someembodiments, the deformation of the target tissue region is determinedbased at least on the set of spatial measurements. In some embodiments,the images of the target tissue region comprise two-dimensional imagesof the target tissue region, wherein the set of spatial measurements ofthe target tissue region is determined based on the two-dimensionalimages of the target tissue region. In some embodiments, the images ofthe target tissue region comprise three-dimensional images of the targettissue region, wherein the set of spatial measurements of the targettissue region is determined based on the three-dimensional images of thetarget tissue region. In some embodiments, the set of spatialmeasurements of the target tissue region are two-dimensional. In someembodiments, the set of spatial measurements of the target tissue regionare two-dimensional, wherein one dimension is normal to the targettissue region. In some embodiments, the set of spatial measurements ofthe target tissue region are three-dimensional.

Viscoelastic Property

In some embodiments, the viscoelastic property of the target tissueregion is determined based at least on the deformation of the targettissue region, the perfusion property of the target tissue region, orboth. In some embodiments, the viscoelastic property comprises aviscosity property, an elastic property, a fluid mechanics property, orany combination thereof. In some embodiments, the viscoelastic propertycomprises a stiffness. In some embodiments, the viscosity propertycorrelates to a rate at which the target tissue deforms under force. Insome embodiments, the elastic property correlates to the deformationdistance under force. In some embodiments, the viscosity property is akinematic viscosity, a dynamic viscosity, or both. In some embodiments,the fluid mechanics property is a flow resistance, a pulse rate, a fluidpressure, a fluid volume, a fluid temperature, a fluid density, or anycombination thereof.

Types of Imaging

FIGS. 5A and 5B show images of a device for obtaining a set of images ofthe target tissue region, without and with a laparoscope, respectively.FIG. 6 shows an image of a connectivity device for transferring the setof images of the target tissue region. FIG. 7 shows an image of a systemfor collecting and transferring the set of images of the target tissueregion

In some embodiments, the set of images comprises a laser speckle image,a Red-Green-Blue (RGB) image, an RGB-Depth image, or any combinationthereof. In some embodiments, the set of images comprises a laserspeckle video, a Red-Green-Blue (RGB) video, an RGB-Depth video, or anycombination thereof. In some embodiments, the RGB-Depth image comprisesan RGB image overlaid with a depth measurement. In some embodiments, thelaser speckle image is a subjective laser speckle image, an objectivelaser speckle image, a near-field laser speckle image, or anycombination thereof. In some embodiments, a subjective laser speckleimage is captured while the sample is directly illuminated the with acoherent light (e.g. a laser beam). In some embodiments, the subjectivelaser speckle image depends on the viewing system parameters, such as,for example: the size of the lens aperture, and the position of theimaging system. In some embodiments, a subjective laser speckle image iscaptured while the sample is indirectly illuminated the with a coherentlight (e.g. a laser beam). In some embodiments, the laser speckle imageis captured by a camera.

In some embodiments, the set of images is obtained while emitting two ormore different wavelengths of light at the target tissue region. In someembodiments, the set of images is obtained while emitting about 10 toabout 1,000 different wavelengths of light at the target tissue region.In some embodiments, per FIG. 3 , the set of images is obtained whileemitting a hyperspectral combination of wavelengths 301, a laserwavelength 302, and a near-infrared wavelength 303. In some embodiments,the set of images of the target issue region and the set spatialmeasurements of the target tissue region are obtained simultaneously inreal time. In some embodiments, the set of images of the target issueregion and the set spatial measurements of the target tissue region areobtained simultaneously in real time as the target issue regionundergoes the deformation. In some embodiments, the set of images of thetarget issue region is obtained in-vitro. In some embodiments, the setof images of the target issue region is obtained in-vivo. In someembodiments, at least one of the set of images of the target issueregion is obtained while the target tissue region undergoes a knowndeformation by a pre-determined force. In some embodiments, a firstimage of the set of images of the target issue region is obtained whilethe target tissue region undergoes a known deformation by apre-determined force. FIG. 11 shows an exemplary setup to capture aspeckle image of the target issue region 1101 while the target tissueregion 1101 undergoes a known deformation by a pre-determined force1103. As shown, a thread 1102 is attached to the target tissue region1101 imparting a known pre-determined force 1103 thereon, while aspeckle image is captured by an image capturing device 1104. As showntherein, the thread 1102 imparts a normal tensile pre-determined force1103 to the target tissue region 1101 via the thread 1102. Additionallyor alternatively, the thread 1102 imparts a normal compressive, or ashear pre-determined force 1103 to the target tissue region 1101.

In some embodiments, the set of images are all captured with the sameorientation between the image capturing device and the target tissue. Insome embodiments, at least a portion of the set of images are allcaptured with the same orientation between the image capturing deviceand the target tissue.

Depth Measurements

In some embodiments, the method further comprises obtaining depthmeasurements from a depth sensor. In some embodiments, the depth sensoris a stereo triangulation sensor, a structured light sensor, a videocamera, a time of flight sensor, an interferometer, a coded aperture, orany combination thereof. In some embodiments, the deformation of thetarget tissue region is further based on the depth measurements. In someembodiments, the spatial measurements are one-dimensional,two-dimensional, or three-dimensional. In some embodiments, thedeformation of the target tissue region comprises a one-dimensionaldeformation, a two-dimensional deformation, a three-dimensionaldeformation, or any combination thereof.

Feedback

In some embodiments, the force is applied by a human operator. In someembodiments, the method further comprises providing a feedback to theoperator. In some embodiments, the method further comprises providing afeedback to the operator based on the determined estimated force appliedon the target tissue region. In some embodiments, the feedback comprisesa visual feedback, an auditory feedback, a haptic feedback, or anycombination thereof. In some embodiments, the visual feedback comprisesa color coded visual feedback, a displayed value, a map, or anycombination thereof corresponding to the estimated force. In someembodiments, a relationship between the estimated force and the feedbackis linear, non-linear, or exponential.

In some embodiments, the force is applied by an autonomous orsemi-autonomous device. In some embodiments, the method furthercomprises providing a control feedback to the autonomous orsemi-autonomous device based on the force applied by the deformedtissue. In some embodiments, the autonomous or semi-autonomous devicealters its treatment based on the control feedback.

Flow Rate and Identification

In some embodiments, the method further comprises determining a fluidflow rate within the target tissue. In some embodiments, the flow rateis based at least on (i) the set of images, (ii) the spatialmeasurements, (iii) the viscoelastic property of the target tissueregion, (iv) the deformation of the target tissue region, or anycombination thereof. In some embodiments, the fluid is blood, sweat,semen, saliva, pus, urine, air, mucus, milk, bile, a hormone, or anycombination thereof. In some embodiments, the fluid flow rate within thetarget tissue is determined by a machine learning algorithm. In someembodiments, the fluid flow rate is determined by a machine learningalgorithm. In some embodiments, the method further comprises determiningan identification of the target tissue based at least on (i) the set ofimages, (ii) the spatial measurements, (iii) the viscoelastic propertyof the target tissue region, (iv) the deformation of the target tissueregion, or any combination thereof. In some embodiments, theidentification of the target tissue is determined by a machine learningalgorithm. In some embodiments, the identification of the target tissueis an identification that the target tissue is cancerous, benign,malignant, or healthy.

Machine Learning

In some embodiments, determining the mechanical property, theviscoelastic property, or both of the target tissue region is performedby a machine learning algorithm. In some embodiments, determining theestimated force applied to the target tissue region is performed by amachine learning algorithm. In some embodiments, the machine learningalgorithm employs a neural network.

Examples of the machine learning algorithms that can be used with theembodiments herein may comprise a regression-based learning algorithm,linear or non-linear algorithms, feed-forward neural network, generativeadversarial network (GAN), or deep residual networks. The machinelearning algorithm may include, for example, an unsupervised learningclassifier, a supervised learning classifier, or a combination thereof.An unsupervised learning classifier may include, for example,clustering, hierarchical clustering, k-means, mixture models, DBSCAN,OPTICS algorithm, anomaly detection, local outlier factor, neuralnetworks, autoencoders, deep belief nets, hebbian learning, generativeadversarial networks, self-organizing map, expectation— maximizationalgorithm (EM), method of moments, blind signal separation techniques,principal component analysis, independent component analysis,non-negative matrix factorization, singular value decomposition, or acombination thereof. A supervised learning classifier may include, forexample, support vector machines, linear regression, logisticregression, linear discriminant analysis, decision trees, k-nearestneighbor algorithm, neural networks, similarity learning, or acombination thereof. In some embodiments, the machine learning algorithmmay comprise a deep learning neural network. The deep learning neuralnetwork may comprise a convolutional neural network (CNN). The CNN mayinclude, for example, U-Net, ImageNet, LeNet-5, AlexNet, ZFNet,GoogleNet, VGGNet, ResNet18 or ResNet, etc.

FIG. 4 shows an exemplary schematic flowchart of a machine learningalgorithm for determining the estimated force applied to the targettissue region. As shown, the exemplary algorithm comprises: receiving afirst input speckle (x0) 401A and a second input speckle (xt) 401B;determining a hidden abstract representation of the first input speckle(h0) 403A and second input speckle (h_t) 403B via an encoder 402,comparing the abstract representations of the first (h0) and secondinput speckles (h_t) 404; and determining an output force 405. In someembodiments, at least one of the first input speckle (h0) 403A and thesecond input speckle (h_t) 403B are captured while a predetermined forceis applied to the target tissue region. As changes between two or morespeckle images can be caused by a motion artifact of the tissue, fluidflow therein, or external forces, the predetermined force applied duringone or more of the speckle images, and the determined perfusionproperty, enables the machine learning algorithms herein todifferentiate changes in the viscoelastic properties of the targettissue region in subsequent speckle images.

In some embodiments, the machine learning algorithm is a supervisedmachine learning algorithm. In some embodiments, the machine learningalgorithms utilized therein employ one or more forms of labels includingbut not limited to human annotated labels and semi-supervised labels.The human annotated labels can be provided by a hand-crafted heuristic.For example, the hand-crafted heuristic can comprise examiningdifferences between images of the target tissue region, spatialmeasurements, or both. The semi-supervised labels can be determinedusing a clustering technique to find images of the target tissue region,spatial measurements, or both similar to those flagged by previous humanannotated labels and previous semi-supervised labels. Thesemi-supervised labels can employ a XGBoost, a neural network, or both.

The distant supervision method can create a large training set seeded bya small hand-annotated training set. The distant supervision method cancomprise positive-unlabeled learning with the training set as the‘positive’ class. The distant supervision method can employ a logisticregression model, a recurrent neural network, or both. The recurrentneural network can be advantageous for Natural Language Processing (NLP)machine learning.

Examples of machine learning algorithms can include a support vectormachine (SVM), a naïve Bayes classification, a random forest, a neuralnetwork, deep learning, or other supervised learning algorithm orunsupervised learning algorithm for classification and regression. Themachine learning algorithms can be trained using one or more trainingdatasets.

In some embodiments, the machine learning algorithm utilizes regressionmodeling, wherein relationships between predictor variables anddependent variables are determined and weighted. In one embodiment, forexample the viscoelastic property can be a dependent variable and isderived from the images of the target tissue region, spatialmeasurements, or both.

In some embodiments, a machine learning algorithm is used to selectcatalogue images and recommend project scope. A non-limiting example ofa multi-variate linear regression model algorithm is seen below:probability=A₀+A₁(X₁)+A₂(X₂)+A₃(X₃)+A₄(X₄)+A₅(X₅)+A₆(X₆)+A₇(X₇) . . .wherein A (A₁, A₂, A₃, A₄, A₅, A₆, A₇, . . . ) are “weights” orcoefficients found during the regression modeling; and X₁ (X₁, X₂, X₃,X₄, X₅, X₆, X₇, . . . ) are data collected from the User. Any number ofA_(i) and X_(i) variable can be included in the model. For example, in anon-limiting example wherein there are 7 X_(i) terms, X₁ is the numberof images, X₂ is the number of spatial measurement, and X₃ is theviscoelastic property of the target tissue region. In some embodiments,the programming language “R” is used to run the model.]

In some embodiments, training comprises multiple steps. In a first step,an initial model is constructed by assigning probability weights topredictor variables. In a second step, the initial model is used to“recommend” the viscoelastic property of the target tissue region. In athird step, the validation module accepts verified data regarding theviscoelastic property of the target tissue region and feeds back theverified data to the renovation probability calculation. At least one ofthe first step, the second step, and the third step can repeat one ormore times continuously or at set intervals.

Method For Training a Neural Network

Another aspect provided herein is a computer-implemented method fortraining a neural network to determine an elastic property of a targetissue region. In some embodiments, per FIG. 2 , the method comprises:generating a first training set 201; training the neural network in afirst stage using the first training set 202; generating a secondtraining set 203; and training the neural network in a second stageusing the second training set 204.

In some embodiments, the first training set comprising a plurality ofsets of set of images. In some embodiments, each set of images comprisesa first speckle image of the target issue region at rest and a secondspeckle image of the target issue region. In some embodiments, thesecond speckle image is captured while the target issue region is beingdeformed. In some embodiments, the second speckle image is capturedwhile the target issue region is being deformed by a known force. Insome embodiments, the second training set comprising the first trainingset and the sets of set of images whose elastic property value wasincorrectly determined after the first stage of training.

In some embodiments, the set of images comprises a subjective set ofimages, an objective set of images, a near-field set of images, or anycombination thereof. In some embodiments, the set of images is obtainedwhile emitting at least 10 different wavelengths of light at the targettissue region. In some embodiments, the set of images is obtained whileemitting about 10 to about 1,000 different wavelengths of light at thetarget tissue region. In some embodiments, the viscoelastic propertycomprises a viscous property, an elastic property, a fluid mechanicsproperty, or any combination thereof. In some embodiments, the spatialmeasurements are one-dimensional, two-dimensional, or three-dimensional.

Alternative Embodiments

In another aspect, the present disclosure provides a method of trackingtissue deformations. The method may comprise: (a) obtaining a scalaroptical flow reading, wherein the scalar optical flow readingcorresponds to one or more laser speckle signals; (b) using said scalaroptical flow reading to determine a pixel-wise motion magnitude estimatefor a tissue region; and (c) integrating said pixel-wise motionmagnitude estimate over time and space to track a deformation of thetissue region. In some embodiments, the one or more laser specklesignals may be associated with, based on, and/or derived from thedeformation of the tissue region. In some embodiments, the one or morelaser speckle signals may be obtained during a deformation of the tissueregion. In some embodiments, the pixel-wise motion magnitude estimatemay comprise a directionless motion estimate. In some cases, the methodmay further comprise combining (i) the pixel-wise motion estimate with(ii) depth and/or RGB-D data of the tissue region to generate apixel-wise displacement map. The pixel-wise displacement map maycomprise a visual or data-based representation of a deformation of atissue region at one or more pixels (or per pixel of an image of thetissue region).

Terms and Definitions

Unless otherwise defined, all technical terms used herein have the samemeaning as commonly understood by one of ordinary skill in the art towhich this disclosure belongs.

As used herein, the singular forms “a,” “an,” and “the” include pluralreferences unless the context clearly dictates otherwise. Any referenceto “or” herein is intended to encompass “and/or” unless otherwisestated.

As used herein, the term “about” in some cases refers to an amount thatis approximately the stated amount.

As used herein, the term “about” refers to an amount that is near thestated amount by 10%, 5%, or 1%, including increments therein.

As used herein, the term “about” in reference to a percentage refers toan amount that is greater or less the stated percentage by 10%, 5%, or1%, including increments therein.

As used herein, the phrases “at least one”, “one or more”, and “and/or”are open-ended expressions that are both conjunctive and disjunctive inoperation. For example, each of the expressions “at least one of A, Band C”, “at least one of A, B, or C”, “one or more of A, B, and C”, “oneor more of A, B, or C” and “A, B, and/or C” means A alone, B alone, Calone, A and B together, A and C together, B and C together, or A, B andC together.

As used herein, the term “perfusion” refers to is a measurement of thepassage of fluid through an organ or a tissue. In some embodiments,perfusion is measured as the rate at which blood is delivered to tissue,or volume of blood per unit time (blood flow) per unit tissue mass. Insome embodiments, perfusion is measured in m³/(s·kg) or ml/min/g.

As used herein, the term “speckle image” refers to a pattern is producedby the mutual interference of a set of incoherent waves. In someembodiments, the waves have the same frequency, having different phasesand amplitudes, which add together to give a resultant wave whoseamplitude varies randomly.

Computing System

Referring to FIG. 12 , a block diagram is shown depicting an exemplarymachine that includes a computer system 1200 (e.g., a processing orcomputing system) within which a set of instructions can execute forcausing a device to perform or execute any one or more of the aspectsand/or methodologies for static code scheduling of the presentdisclosure. The components in FIG. 12 are examples only and do not limitthe scope of use or functionality of any hardware, software, embeddedlogic component, or a combination of two or more such componentsimplementing particular embodiments.

Computer system 1200 may include one or more processors 1201, a memory1203, and a storage 1208 that communicate with each other, and withother components, via a bus 1240. The bus 1240 may also link a display1232, one or more input devices 1233 (which may, for example, include akeypad, a keyboard, a mouse, a stylus, etc.), one or more output devices1234, one or more storage devices 1235, and various tangible storagemedia 1236. All of these elements may interface directly or via one ormore interfaces or adaptors to the bus 1240. For instance, the varioustangible storage media 1236 can interface with the bus 1240 via storagemedium interface 1226. Computer system 1200 may have any suitablephysical form, including but not limited to one or more integratedcircuits (ICs), printed circuit boards (PCBs), mobile handheld devices(such as mobile telephones or PDAs), laptop or notebook computers,distributed computer systems, computing grids, or servers.

Computer system 1200 includes one or more processor(s) 1201 (e.g.,central processing units (CPUs) or general purpose graphics processingunits (GPGPUs)) that carry out functions. Processor(s) 1201 optionallycontains a cache memory unit 1202 for temporary local storage ofinstructions, data, or computer addresses. Processor(s) 1201 areconfigured to assist in execution of computer readable instructions.Computer system 1200 may provide functionality for the componentsdepicted in FIG. 12 as a result of the processor(s) 1201 executingnon-transitory, processor-executable instructions embodied in one ormore tangible computer-readable storage media, such as memory 1203,storage 1208, storage devices 1235, and/or storage medium 1236. Thecomputer-readable media may store software that implements particularembodiments, and processor(s) 1201 may execute the software. Memory 1203may read the software from one or more other computer-readable media(such as mass storage device(s) 1235, 1236) or from one or more othersources through a suitable interface, such as network interface 1220.The software may cause processor(s) 1201 to carry out one or moreprocesses or one or more steps of one or more processes described orillustrated herein. Carrying out such processes or steps may includedefining data structures stored in memory 1203 and modifying the datastructures as directed by the software.

The memory 1203 may include various components (e.g., machine readablemedia) including, but not limited to, a random access memory component(e.g., RAM 1204) (e.g., static RAM (SRAM), dynamic RAM (DRAM),ferroelectric random access memory (FRAM), phase-change random accessmemory (PRAM), etc.), a read-only memory component (e.g., ROM 1205), andany combinations thereof. ROM 1205 may act to communicate data andinstructions unidirectionally to processor(s) 1201, and RAM 1204 may actto communicate data and instructions bidirectionally with processor(s)1201. ROM 1205 and RAM 1204 may include any suitable tangiblecomputer-readable media described below. In one example, a basicinput/output system 1206 (BIOS), including basic routines that help totransfer information between elements within computer system 1200, suchas during start-up, may be stored in the memory 1203.

Fixed storage 1208 is connected bidirectionally to processor(s) 1201,optionally through storage control unit 1207. Fixed storage 1208provides additional data storage capacity and may also include anysuitable tangible computer-readable media described herein. Storage 1208may be used to store operating system 1209, executable(s) 1210, data1211, applications 1212 (application programs), and the like. Storage1208 can also include an optical disk drive, a solid-state memory device(e.g., flash-based systems), or a combination of any of the above.Information in storage 1208 may, in appropriate cases, be incorporatedas virtual memory in memory 1203.

In one example, storage device(s) 1235 may be removably interfaced withcomputer system 1200 (e.g., via an external port connector (not shown))via a storage device interface 1225. Particularly, storage device(s)1235 and an associated machine-readable medium may provide non-volatileand/or volatile storage of machine-readable instructions, datastructures, program modules, and/or other data for the computer system1200. In one example, software may reside, completely or partially,within a machine-readable medium on storage device(s) 1235. In anotherexample, software may reside, completely or partially, withinprocessor(s) 1201.

Bus 1240 connects a wide variety of subsystems. Herein, reference to abus may encompass one or more digital signal lines serving a commonfunction, where appropriate. Bus 1240 may be any of several types of busstructures including, but not limited to, a memory bus, a memorycontroller, a peripheral bus, a local bus, and any combinations thereof,using any of a variety of bus architectures. As an example and not byway of limitation, such architectures include an Industry StandardArchitecture (ISA) bus, an Enhanced ISA (EISA) bus, a Micro ChannelArchitecture (MCA) bus, a Video Electronics Standards Association localbus (VLB), a Peripheral Component Interconnect (PCI) bus, a PCI-Express(PCI-X) bus, an Accelerated Graphics Port (AGP) bus, HyperTransport(HTX) bus, serial advanced technology attachment (SATA) bus, and anycombinations thereof.

Computer system 1200 may also include an input device 1233. In oneexample, a user of computer system 1200 may enter commands and/or otherinformation into computer system 1200 via input device(s) 1233. Examplesof an input device(s) 1233 include, but are not limited to, analpha-numeric input device (e.g., a keyboard), a pointing device (e.g.,a mouse or touchpad), a touchpad, a touch screen, a multi-touch screen,a joystick, a stylus, a gamepad, an audio input device (e.g., amicrophone, a voice response system, etc.), an optical scanner, a videoor still image capture device (e.g., a camera), and any combinationsthereof. In some embodiments, the input device is a Kinect, Leap Motion,or the like. Input device(s) 1233 may be interfaced to bus 1240 via anyof a variety of input interfaces 1223 (e.g., input interface 1223)including, but not limited to, serial, parallel, game port, USB,FIREWIRE, THUNDERBOLT, or any combination of the above.

In particular embodiments, when computer system 1200 is connected tonetwork 1230, computer system 1200 may communicate with other devices,specifically mobile devices and enterprise systems, distributedcomputing systems, cloud storage systems, cloud computing systems, andthe like, connected to network 1230. Communications to and from computersystem 1200 may be sent through network interface 1220. For example,network interface 1220 may receive incoming communications (such asrequests or responses from other devices) in the form of one or morepackets (such as Internet Protocol (IP) packets) from network 1230, andcomputer system 1200 may store the incoming communications in memory1203 for processing. Computer system 1200 may similarly store outgoingcommunications (such as requests or responses to other devices) in theform of one or more packets in memory 1203 and communicated to network1230 from network interface 1220. Processor(s) 1201 may access thesecommunication packets stored in memory 1203 for processing.

Examples of the network interface 1220 include, but are not limited to,a network interface card, a modem, and any combination thereof. Examplesof a network 1230 or network segment 1230 include, but are not limitedto, a distributed computing system, a cloud computing system, a widearea network (WAN) (e.g., the Internet, an enterprise network), a localarea network (LAN) (e.g., a network associated with an office, abuilding, a campus or other relatively small geographic space), atelephone network, a direct connection between two computing devices, apeer-to-peer network, and any combinations thereof. A network, such asnetwork 1230, may employ a wired and/or a wireless mode ofcommunication. In general, any network topology may be used.

Information and data can be displayed through a display 1232. Examplesof a display 1232 include, but are not limited to, a cathode ray tube(CRT), a liquid crystal display (LCD), a thin film transistor liquidcrystal display (TFT-LCD), an organic liquid crystal display (OLED) suchas a passive-matrix OLED (PMOLED) or active-matrix OLED (AMOLED)display, a plasma display, and any combinations thereof. The display1232 can interface to the processor(s) 1201, memory 1203, and fixedstorage 1208, as well as other devices, such as input device(s) 1233,via the bus 1240. The display 1232 is linked to the bus 1240 via a videointerface 1222, and transport of data between the display 1232 and thebus 1240 can be controlled via the graphics control 1221. In someembodiments, the display is a video projector. In some embodiments, thedisplay is a head-mounted display (HMD) such as a VR headset. In furtherembodiments, suitable VR headsets include, by way of non-limitingexamples, HTC Vive, Oculus Rift, Samsung Gear VR, Microsoft HoloLens,Razer OSVR, FOVE VR, Zeiss VR One, Avegant Glyph, Freefly VR headset,and the like. In still further embodiments, the display is a combinationof devices such as those disclosed herein.

In addition to a display 1232, computer system 1200 may include one ormore other peripheral output devices 1234 including, but not limited to,an audio speaker, a printer, a storage device, and any combinationsthereof. Such peripheral output devices may be connected to the bus 1240via an output interface 1224. Examples of an output interface 1224include, but are not limited to, a serial port, a parallel connection, aUSB port, a FIREWIRE port, a THUNDERBOLT port, and any combinationsthereof.

In addition, or as an alternative, computer system 1200 may providefunctionality as a result of logic hardwired or otherwise embodied in acircuit, which may operate in place of or together with software toexecute one or more processes or one or more steps of one or moreprocesses described or illustrated herein. Reference to software in thisdisclosure may encompass logic, and reference to logic may encompasssoftware. Moreover, reference to a computer-readable medium mayencompass a circuit (such as an IC) storing software for execution, acircuit embodying logic for execution, or both, where appropriate. Thepresent disclosure encompasses any suitable combination of hardware,software, or both; and

Those of skill in the art will appreciate that the various illustrativelogical blocks, modules, circuits, and algorithm steps described inconnection with the embodiments disclosed herein may be implemented aselectronic hardware, computer software, or combinations of both. Toclearly illustrate this interchangeability of hardware and software,various illustrative components, blocks, modules, circuits, and stepshave been described above generally in terms of their functionality.

The various illustrative logical blocks, modules, and circuits describedin connection with the embodiments disclosed herein may be implementedor performed with a general purpose processor, a digital signalprocessor (DSP), an application specific integrated circuit (ASIC), afield programmable gate array (FPGA) or other programmable logic device,discrete gate or transistor logic, discrete hardware components, or anycombination thereof designed to perform the functions described herein.A general purpose processor may be a microprocessor, but in thealternative, the processor may be any conventional processor,controller, microcontroller, or state machine. A processor may also beimplemented as a combination of computing devices, e.g., a combinationof a DSP and a microprocessor, a plurality of microprocessors, one ormore microprocessors in conjunction with a DSP core, or any other suchconfiguration.

The steps of a method or algorithm described in connection with theembodiments disclosed herein may be embodied directly in hardware, in asoftware module executed by one or more processor(s), or in acombination of the two. A software module may reside in RAM memory,flash memory, ROM memory, EPROM memory, EEPROM memory, registers, harddisk, a removable disk, a CD-ROM, or any other form of storage mediumknown in the art. An exemplary storage medium is coupled to theprocessor such the processor can read information from, and writeinformation to, the storage medium. In the alternative, the storagemedium may be integral to the processor. The processor and the storagemedium may reside in an ASIC. The ASIC may reside in a user terminal. Inthe alternative, the processor and the storage medium may reside asdiscrete components in a user terminal.

In accordance with the description herein, suitable computing devicesinclude, by way of non-limiting examples, server computers, desktopcomputers, laptop computers, notebook computers, sub-notebook computers,netbook computers, netpad computers, set-top computers, media streamingdevices, handheld computers, Internet appliances, mobile smartphones,tablet computers, personal digital assistants, video game consoles, andvehicles. Those of skill in the art will also recognize that selecttelevisions, video players, and digital music players with optionalcomputer network connectivity are suitable for use in the systemdescribed herein. Suitable tablet computers, in various embodiments,include those with booklet, slate, and convertible configurations, knownto those of skill in the art.

In some embodiments, the computing device includes an operating systemconfigured to perform executable instructions. The operating system is,for example, software, including programs and data, which manages thedevice's hardware and provides services for execution of applications.Those of skill in the art will recognize that suitable server operatingsystems include, by way of non-limiting examples, FreeBSD, OpenBSD,NetBSD®, Linux, Apple® Mac OS X Server®, Oracle® Solaris®, WindowsServer®, and Novell® NetWare®. Those of skill in the art will recognizethat suitable personal computer operating systems include, by way ofnon-limiting examples, Microsoft® Windows®, Apple® Mac OS X®, UNIX®, andUNIX-like operating systems such as GNU/Linux®. In some embodiments, theoperating system is provided by cloud computing. Those of skill in theart will also recognize that suitable mobile smartphone operatingsystems include, by way of non-limiting examples, Nokia® Symbian® OS,Apple® iOS®, Research In Motion® BlackBerry OS®, Google® Android®,Microsoft® Windows Phone® OS, Microsoft® Windows Mobile® OS, Linux®, andPalm® WebOS®. Those of skill in the art will also recognize thatsuitable media streaming device operating systems include, by way ofnon-limiting examples, Apple TV®, Roku®, Boxee®, Google TV®, GoogleChromecast®, Amazon Fire®, and Samsung® HomeSync®. Those of skill in theart will also recognize that suitable video game console operatingsystems include, by way of non-limiting examples, Sony®, PS3®, Sony®,PS4®, Microsoft® Xbox 360®, Microsoft Xbox One, Nintendo® Wii®,Nintendo® Wii®, and Ouya®.

Non-Transitory Computer Readable Storage Medium

In some embodiments, the platforms, systems, media, and methodsdisclosed herein include one or more non-transitory computer readablestorage media encoded with a program including instructions executableby the operating system of an optionally networked computing device. Infurther embodiments, a computer readable storage medium is a tangiblecomponent of a computing device. In still further embodiments, acomputer readable storage medium is optionally removable from acomputing device. In some embodiments, a computer readable storagemedium includes, by way of non-limiting examples, CD-ROMs, DVDs, flashmemory devices, solid state memory, magnetic disk drives, magnetic tapedrives, optical disk drives, distributed computing systems includingcloud computing systems and services, and the like. In some cases, theprogram and instructions are permanently, substantially permanently,semi-permanently, or non-transitorily encoded on the media.

Computer Program

In some embodiments, the platforms, systems, media, and methodsdisclosed herein include at least one computer program, or use of thesame. A computer program includes a sequence of instructions, executableby one or more processor(s) of the computing device's CPU, written toperform a specified task. Computer readable instructions may beimplemented as program modules, such as functions, objects, ApplicationProgramming Interfaces (APIs), computing data structures, and the like,that perform particular tasks or implement particular abstract datatypes. In light of the disclosure provided herein, those of skill in theart will recognize that a computer program may be written in variousversions of various languages.

The functionality of the computer readable instructions may be combinedor distributed as desired in various environments. In some embodiments,a computer program comprises one sequence of instructions. In someembodiments, a computer program comprises a plurality of sequences ofinstructions. In some embodiments, a computer program is provided fromone location. In other embodiments, a computer program is provided froma plurality of locations. In various embodiments, a computer programincludes one or more software modules. In various embodiments, acomputer program includes, in part or in whole, one or more webapplications, one or more mobile applications, one or more standaloneapplications, one or more web browser plug-ins, extensions, add-ins, oradd-ons, or combinations thereof.

Software Modules

In some embodiments, the platforms, systems, media, and methodsdisclosed herein include software, server, and/or database modules, oruse of the same. In view of the disclosure provided herein, softwaremodules are created by techniques known to those of skill in the artusing machines, software, and languages known to the art. The softwaremodules disclosed herein are implemented in a multitude of ways. Invarious embodiments, a software module comprises a file, a section ofcode, a programming object, a programming structure, or combinationsthereof. In further various embodiments, a software module comprises aplurality of files, a plurality of sections of code, a plurality ofprogramming objects, a plurality of programming structures, orcombinations thereof. In various embodiments, the one or more softwaremodules comprise, by way of non-limiting examples, a web application, amobile application, and a standalone application. In some embodiments,software modules are in one computer program or application. In otherembodiments, software modules are in more than one computer program orapplication. In some embodiments, software modules are hosted on onemachine. In other embodiments, software modules are hosted on more thanone machine. In further embodiments, software modules are hosted on adistributed computing platform such as a cloud computing platform. Insome embodiments, software modules are hosted on one or more machines inone location. In other embodiments, software modules are hosted on oneor more machines in more than one location.

Databases

In some embodiments, the platforms, systems, media, and methodsdisclosed herein include one or more databases, or use of the same. Inview of the disclosure provided herein, those of skill in the art willrecognize that many databases are suitable for storage and retrieval ofimage, flow rate, force, elastic, perfusion, viscoelastic information,or any combination thereof. In various embodiments, suitable databasesinclude, by way of non-limiting examples, relational databases,non-relational databases, object oriented databases, object databases,entity-relationship model databases, associative databases, and XMLdatabases. Further non-limiting examples include SQL, PostgreSQL, MySQL,Oracle, DB2, and Sybase. In some embodiments, a database isinternet-based. In further embodiments, a database is web-based. Instill further embodiments, a database is cloud computing-based. In aparticular embodiment, a database is a distributed database. In otherembodiments, a database is based on one or more local computer storagedevices.

1-99. (canceled)
 100. A computer-implemented method for determining anestimated force applied on a target tissue region, the methodcomprising: (a) obtaining a set of images of the target tissue region;(b) determining a perfusion property, a set of spatial measurements, orboth of the target tissue region based at least on the set of images;(c) determining a deformation of the target tissue region based at leaston the set of spatial measurements; (d) determining a viscoelasticproperty of the target tissue region based at least on the deformationof the target tissue region, the perfusion property of the target tissueregion, or both; and (e) determining the estimated force applied on thetarget tissue region based at least on the viscoelastic property of thetarget tissue region.
 101. The method of claim 100, wherein the set ofimages comprises a laser speckle image, an RGB image, an RGB-Depthimage, or any combination thereof.
 102. The method of claim 101, whereinthe laser speckle image is a subjective laser speckle image, anobjective laser speckle image, a near-field laser speckle image, or anycombination thereof.
 103. The method of claim 100, wherein the set ofimages is obtained while emitting two or more different wavelengths oflight at the target tissue region.
 104. The method of claim 100, whereinthe set of images of the target issue region and the set of spatialmeasurements of the target tissue region are obtained simultaneously inreal time as the target issue region undergoes the deformation.
 105. Themethod of claim 100, wherein the set of images of the target issueregion is obtained in-vitro.
 106. The method of claim 100, wherein theset of images of the target issue region is obtained in-vivo.
 107. Themethod of claim 100, wherein at least one of the set of images of thetarget issue region is obtained while the target tissue region undergoesa known deformation by a pre-determined force.
 108. The method of claim100, wherein determining the mechanical property, the viscoelasticproperty, or both of the target tissue region is performed by a machinelearning algorithm.
 109. The method of claim 100, further comprisingobtaining depth measurements from a depth sensor, and wherein thedeformation of the target tissue region is further based on the depthmeasurements.
 110. The method of claim 100, wherein the deformation ofthe target tissue region comprises a one-dimensional deformation, atwo-dimensional deformation, a three-dimensional deformation, or anycombination thereof.
 111. The method of claim 100, wherein determiningthe estimated force applied to the target tissue region is performed bya machine learning algorithm.
 112. The method of claim 100, wherein aforce is applied by a human operator, and wherein the method furthercomprises providing a feedback to the operator based on the determinedestimated force applied on the target tissue region.
 113. The method ofclaim 100, wherein a force is applied by an autonomous orsemi-autonomous device, and wherein the method further comprisesproviding a control feedback to the autonomous or semi-autonomous devicebased on the force applied by the deformed tissue.
 114. The method ofclaim 100, further comprising determining a fluid flow rate of a fluidwithin the target tissue based at least on (i) the set of images, (ii)the spatial measurements, (iii) the viscoelastic property of the targettissue region, (iv) the deformation of the target tissue region, or anycombination thereof.
 115. The method of claim 114, wherein the fluid isblood, sweat, semen, saliva, pus, urine, air, mucus, milk, bile, ahormone, or any combination thereof.
 116. The method of claim 114,wherein the fluid flow rate within the target tissue is determined by amachine learning algorithm.
 117. The method of claim 114, wherein thefluid flow rate is determined by a machine learning algorithm.
 118. Themethod of claim 100, further comprising determining an identification ofthe target tissue based at least on (i) the set of images, (ii) thespatial measurements, (iii) the viscoelastic property of the targettissue region, (iv) the deformation of the target tissue region, or anycombination thereof.
 119. The method of claim 118, wherein theidentification of the target tissue is determined by a machine learningalgorithm.